Evaluations of Automated Scoring Systems in Practice

Guided Retrieval Practice of Educational Materials Using Automated Scoring

Evaluating the performance of Automated Text Scoring systems

Various measures have been used to evaluate the effectiveness of automated text scoring (ATS) systems with respect to a human gold standard. However, there is no systematic study comparing the efficacy of these metrics under different experimental conditions. In this paper we first argue that measures of agreement are more appropriate than measures of association (i.e., correlation) for measuri...

Semi-quantitative segmental perfusion scoring in myocardial perfusion SPECT: visual vs. automated analysis

Introduction: It is recommended that the physician apply at least a semi-quantitative segmental scoring system in myocardial perfusion SPECT.  We aimed to assess the agreement between automated semi-quantitative analysis using QPS (quantitative Perfusion SPECT) software and visual approach for calculation of summed stress  score (SSS), summed rest score (SRS) and summed difference score (SDS). ...

Beyond Automated Essay Scoring

The ability to communicate in natural language has long been considered a defining characteristic of human intelligence. Furthermore, we hold our ability to express ideas in writing as a pinnacle of this uniquely human language facility—it defies formulaic or algorithmic specification. So it comes as no surprise that attempts to devise computer programs that evaluate writing are often met with ...

Automated Scoring of Crystallization Trials

Recently, the use of robotics and parallel techniques for protein production and crystallization is becoming commonplace among structural genomics initiatives, whose contributions amount to 73% of newly solved structures each year. Despite the strides made in increasing physical experimental throughput, the act of finding just a few crystals among potentially thousands of crystallization experi...